Traditionally, file uploads are made from a single client to a single remote destination computer and are bound by the performance of the network which connects them. For example, throughput using Transmission Control Protocol (TCP) as an underlying transport protocol can be limited by poorly tuned window sizes and high latency.
Grid computing is the virtualization of distributed computing resources such as processing power, networks, bandwidth, and storage capacity to create a single system image, granting users and applications seamless access to vast IT capabilities. A source for more information about grid computing is found in “The Physiology of the Grid—an open grid services architecture for distributed systems integration” by Foster, Kesselman, Nick and Tuecke.
Further information regarding grid computing may be found in a paper entitled “Grid Computing Distribution Using Network Processors” by Llevist and Bengsson. The paper discloses a concept for a new computing model for distribution of both code and data to nodes of a network (grid), wherein the routers of the network determine which nodes should be selected to execute the code based on the knowledge collected by the routers. The subject matter of this document is incorporated herein by reference.
It is known in the prior art to upload data using multiple upload channels. For example, File Transfer Protocol (FTP) and Secure File Transfer Protocol (SFTP) each allow clients to upload portions of a file which enables one to upload the portions concurrently. The concurrent upload, however, requires explicit support in the underlying file system for simultaneous file updates. The client is charged with coordination of the multiple uploads and ensuring the correctness of the uploaded file. For example, the client needs to ensure unique portions of the file are uploaded by each channel, and validation of the upload, if desired, could only be achieved by downloading the entire file and comparing it to the source file. Finally, multi-site file upload would only be possible when the multiple sites are connected to the same file store.
A method, system, and computer program product allowing efficient and reliable simultaneous uploading of data over multiple channels while also providing re-start capability would be desirable.